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Abstract 

The  purpose  of  this  paper  is  to  describe  the  unique  cognitive  processes  that  are  employed 
to  optimize  collaborative  team  decision  making  in  a  geographically  distributed  and  time 
delayed  situation.  The  cognitive  processes  will  be  illustrated  within  the  context  of  a 
structural  model  of  team  collaboration.  The  team  collaboration  model  has  four  unique  but 
interdependent  stages  of  team  collaboration.  The  stages  are:  Team  Knowledge 
Construction,  Collaboratiye  Team  Problem  Solving,  Team  Consensus,  and  Product 
Evaluation  &  Revision.  The  stages  are  not  strictly  sequential  but  are  very  dynamic 
throughout  the  collaboration  process.  The  cognitive  processes  are  represented  at  four 
levels:  Meta-cognition,  which  guides  the  overall  problem  solving  process,  the 
Information  Processing  Tasks  required  by  the  team  to  complete  each  collaboration  stage, 
the  Knowledge  Required  to  support  the  information  processing  tasks  and  the 
Communication  Mechanisms  for  Knowledge  Building  and  Information  Processing. 

Results  from  the  empirical  team  collaboration  studies  indicate  significant  differences  in 
the  cognitive  processes  and  sub-processes  across  the  four  collaboration  stages  for  the 
face-to-face  teams  compared  to  asynchronous,  distributed  teams.  Understanding  these 
differences  will  facilitate  the  design  of  collaboration  tools  for  asynchronous,  distributed 
teams  and  will  provide  for  more  effective  and  timely  collaborative  decision-making. 

Introduction 

The  structural  model  of  team  collaboration  described  in  this  paper  emphasizes  the 
cognitive  aspects  of  the  collaboration  process.  The  model  includes  the  domain 
characteristics,  collaboration  stages,  meta-cognitive  processes,  information  processing 
tasks,  knowledge  required  for  each  information  task  and  the  communication  mechanisms 
for  knowledge  building  and  information  tasks.  There  have  been  numerous  models  of 
team  collaboration  (Orasanu  &  Salas,  1992;  Rogers  &  Ellis,  1994;  Stahl,  2000; 
McNeese,  Rentsch,  Perusich,  2000;  Hurley,  2002;  Noble,  2002)  each  focusing  on  various 
aspects  of  collaboration  while  describing  those  aspects  at  different  levels  of  detail. 
However,  for  a  model  of  collaboration  to  be  an  effective  mechanism  for  understanding 
the  operating  cognitive  mechanisms  underlying  collaborative  team  behavior  the  model 
needs  to  be  defined  at  a  level  of  granularity,  which  covers  all  the  major  components  and 
mechanisms  of  team  collaboration.  The  approach  to  describing  the  current  model  is  as 
follows:  (1)  define  the  problem  domain  for  the  model,  (2)  define  all  the  various 
collaboration  stages  that  a  team  goes  through  to  solve  the  problem,  (3)  define  the  meta- 
cognitive  processes  that  guide  team  collaboration,  (4)  define  and  describe  the  information 
processing  components  that  the  team  performs  to  achieve  each  collaboration  stage,  (5) 
define  the  knowledge  required  to  achieve  each  information  processing  component  and  (6) 
define  the  communication  mechanisms  used  by  the  team  to  build  the  necessary 
knowledge  along  with  supporting  the  information  processing.The  cognitive  mechanisms 
in  the  model  are  described  at  a  macro  level  (i.e.  meta-cognition, information  processing, 
knowledge  building  and  communication  mechanisms)  rather  than  at  the  micro  level  (i.e. 
neural-cognitive).  The  reason  for  describing  the  model’s  cognitive  processes  at  the  macro 
level  is  driven  by  our  limited  understanding  of  how  teams  think  during  collaborative 
problem  solving  in  asynchronous,  distributed  environments.  In  addition,  our  current 
metrics  for  measuring,  at  the  micro  level,  the  cognitive  mechanisms  teams  use  to  solve 
collaborative  problems  are  still  pre-mature  and  need  further  refinement.  The  model’s 
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macro  level  definition  of  the  cognitive  processes  permits  empirical  assessment  of  these 
cognitive  processes  using  our  current  measurement  techniques  (e.g.  verbal  protocol, 
communication  analysis). 

Before  getting  into  a  detailed  model  description  it  is  important  to  understand  the  scope  of 
the  problem  area  that  the  collaborative  model  will  be  addressing.  Figure  1  illustrates  the 
major  factors  impacting  military  collaborative  teams.  These  factors  include  the 
Collaborative  Problem  Environment,  Operational  Tasks,  Collaborative  Situation 
Parameters,  and  Team  Types.  The  collaborative  problem  environment  has  grown  in 
complexity  over  the  past  decade  (Jensen,  2002).  The  military  problems  are  becoming 
more  complex  requiring  teams  to  address  the  problems.  In  addition,  problems  are 
addressed  at  an  international  level  requiring  agile  coalition  operations.  Developments  in 
information  and  communication  technologies  have  provided  greater  communication 
between  coalitions  but  information  overload  is  still  a  problem  due  to  a  lack  of 
information  management  (Information  Management  Strategic  Plan,  1999).  There  are 
many  operational  tasks,  which  involve  team  collaboration  (Jensen,  2002;  Joint  Vision 
2010,  2002).  However,  to  gain  an  understanding  of  the  team  collaboration  process  the 
model  will  focus  on  two  operational  tasks:  (1)  team  decision-making,  course  of  action 
selection,  and  (2)  Intelligence  analysis  (team  data  processing).  During  team  collaboration 
there  are  various  parameters  that  can  influence  collaboration  performance  (Warner, 
Letsky  &  Cowen,  2003).  However,  the  collaborative  situation  parameters  listed  in  Figure 
1  were  chosen  as  the  critical  parameters  to  focus  our  collaboration  domain  because  of 
their  significance  to  current  military  requirements  (Jensen,  2002).  The  final  factor  is  team 
types,  which  of  all  the  factors  has  the  most  number  of  different  categories  that  can 
influence  collaboration  performance.  The  7  team  type  characteristics  listed  in  Figure  1 
were  selected  based  on  the  common  characteristics  of  today’s  military  collaborative 
teams  (Jensen,  2002).  In  summary,  the  problem  area  domain  for  our  structural  model  of 
team  collaboration  can  be  defined  by  the  respective  characteristics  under  operational 
tasks,  collaborative  situation  parameters,  and  team  types  specified  in  Figure  1. 

In  addition  to  understanding  the  model’s  problem  domain,  it  is  also  important  to 
understand  how  current  theories  of  cognitive  psychology  impact  the  model  definition  and 
design.  One  of  the  key  issues  in  developing  a  collaboration  model  is  to  understand  the 
cognitive  processes  that  team  member’s  use  as  they  perform  the  various  stages  of 
collaboration.  To  understand  these  cognitive  processes  it  is  necessary  to  examine  the 
various  theories  of  cognitive  psychology.  Early  philosophers  such  as  Descartes  (1641) 
and  Kant  (1781)  have  examined  basic  questions  of  the  origin  of  knowledge  and  human 
thought.  However,  it  was  not  until  after  the  failure  of  behaviorism  (Skinner,  1985,1989) 
to  explain  internal  representations  that  cognitive  psychology  emerged.  A  multitude  of 
theories  of  human  cognition  began  to  develop.  Several  theories  (Turing,  1936;  Weiner, 
1948;  Shannon,  1949;  Wickens,  1992)  explain  human  cognition  in  terms  of  an 
information-processing  mode/,  which  focus  on  information  representation,  processing 
and  computation.  Other  theories  (Chomsky,  1957;  Cooke,  2003)  believe  that  the 
development  of  knowledge  and  the  processes  of  thought  are  achieved  through  human 
language.  Piaget’s  (1970b)  focused  on  developmental  biology,  which  described  not  only 
the  different  components  of  human  cognition  but  also  the  developmental  stages  of 


3 


Collaborative  Challenges 

Increasing  problem  complexity-  team  effort  needed 
IT/Communications  technology  widening  accessibility  of  contributors 
Problems  addressed  at  international  level  -  coalitions  required 
Defense  Transformation  to  agile  and  coalition  operations 
Information  overload  condition 


Operational  Tasks: 

*  Team  decision  making,  COA  selection 

*  Develop  shared  understanding 

*  Intell  analysis  (team  data  processing) 


Collaborative  Situation  Parameters: 

*  time  pressure 

*  information  /  knowledge  uncertainty 

*  dynamic  information 

*  large  amount  of  knowledge 
(cognitive  overload) 

*  human-agent  interfaces 


*  asynchronous 

*  distributed 

*  culturally  diverse 

*  heterogeneous  knowledge 


Team  Types 


*  unique  roles 

*  command  structure 

(  hierarchical  vs  flat) 

*  rotating  team  members 


Figure  1:  Major  factors  influencing  military  collaborative  teams 


cognition.  As  computer  science  matured,  several  theories  developed  (Newell  &  Simon, 
1956;  Feigenbaum  &  Feldman,  1963;  Anderson,  1993;  Minsky,  1997)  that  explain  human 
cognition  in  terms  of  a  computer  computational  model.  These  computational  models 
varied  in  how  they  explained  cognition  ranging  from  computational  logic,  production 
rules  (e.g.  If  X  Then  Y),  to  frames  (data  structures).  Other  theories  (Rumelhart,  1990; 
Churchland,  1989;  Rosenburg,  1988)  emphasize  physiology  in  understanding  human 
cognition.  These  theories  use  physiological  neural  networks  represented  in 
computational  models  to  explain  cognition  and  its  processes.  Davidson,  Deuser  & 
Sternberg  (1994)  proposed  a  theory  of  meta-cognition,  which  is  knowledge  of  one’s  own 
cognitive  processes,  in  explaining  how  human  cognitive  processes  are  used  in  problem 
solving.  Each  theory  has  provided  unique  insight  and  empirical  data  to  explain  various 
aspects  of  human  cognition  and  its  processes.  Current  research  in  human  cognition  uses  a 
multidisciplinary  approach  by  considering  ideas  from  all  the  relevant  disciplines 
including  philosophy,  psychology,  linguistics,  computer  science,  anthropology,  cognitive 
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neuroscience,  and  artificial  intelligence.  However,  even  with  this  multidisciplinary 
approach  there  is  no  generally  recognized  unified  theory  of  human  cognition.  This  lack  of 
a  unified  theory  is  partially  due  to  the  recent  integration  of  the  various  disciplines  in 
studying  human  cognition  along  with  insufficient  objective  metrics  to  measure  the 
cognitive  processes.  The  challenge  in  representing  the  cognitive  components  in  a 
collaboration  model  is  deciding  what  theoretical  approach(s)  to  utilize  during  initial 
model  development.  There  is  insufficient  empirical  research  describing  the  actual 
cognitive  processes  that  teams  use  during  collaborative  problem  solving  especially  under 
conditions  specified  in  Figure  1 .  As  a  result  of  the  current  state  of  the  various  theories  of 
human  cognition,  the  cognitive  components  of  the  team  collaboration  model  shall  be 
described  using  current  theories  of  meta-cognition,  human  information-processing  and 
human  communication.  These  theoretical  approaches  were  selected  based  on  our  current 
ability  to  measure  how  teams  use  communication  (i.e.  written  and  verbal)  to  process 
information  and  build  knowledge  during  collaborative  problem  solving.  This  level  of 
granularity  in  describing  the  cognitive  processes  in  team  collaboration  permits  empirical 
evaluation  of  the  various  components  of  the  model.  However,  this  level  of  granularity 
will  not  provide  a  comprehensive  view  (e.g.  neural  components)  of  the  cognitive 
processes  used  in  team  collaboration.  A  detail  description  of  the  model’s  cognitive 
components  and  processes  are  presented  under  the  model  section  of  the  paper. 

In  describing  the  team  collaboration  model  it  is  important  to  understand  what  the  term 
model  means  along  with  understanding  the  objectives  of  the  model.  A  model  is  not  the 
system  or  process  itself  but  an  abstract  representation  that  enables  researchers  to  predict 
behavior  and  to  test  hypotheses.  Models  may  be  mathematical,  physical,  or  structural.  For 
example,  information  theory  and  signal  detection  theory  are  mathematical  models;  neuro- 
anatomical  models  are  examples  of  physical  models,  while  information-processing 
models  are  examples  of  structural  models.  Due  to  the  lack  of  sufficient  objective  metrics 
for  measuring  cognition,  most  models  in  cognitive  science  are  either  structural  or 
mathematical  not  physical  models.  The  objectives  of  the  proposed  team  collaboration 
model  are:  (1)  to  understand  the  cognitive  mechanisms  and  their  relationships  during 
team  collaborative  problem  solving,  (2)  to  provide  a  model-based  approach  to 
experimentation  of  team  collaboration,  (3)  to  identify  and  prioritize  required  areas  of 
research  in  team  collaboration,  and  (4)  to  serve  as  a  design  guideline  for  an  agent-based 
support  tool  for  team  collaboration. 

Structural  Model  of  Team  Collaboration 

Figure  2  presents  the  structural  model  of  team  collaboration.  The  model’s  domain  is 
defined  by  the  problem  area  characteristics,  which  were  described  earlier  in  this  paper. 
The  model  consists  of  general  inputs  (e.g.  task  description,  team  roles),  collaboration 
stages  that  the  team  goes  through  during  the  problem-solving  task,  the  cognitive 
processes  used  by  the  team  and  final  team  output(s)  (e.g.  selected  course  of  action, 
recommendations).  The  four  cognitive  processes  include:  (1)  the  meta-cognitive 
processes,  (2)  the  information  processing  tasks,  and  (3)  the  knowledge  required  to 
support  the  information  tasks,  and  (4)  the  communication  mechanisms  for  knowledge 
building  and  information  processing. 
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STRUCTURAL  MODEL  OF  TEAM  COLLABORATION 


Problem  Area 
Characteristics 


•  asynchronous 

•  distributed 

•  culturally  diverse 

•  heterogeneous  knowledge 

•  unique  roles 

•  command  structure 
(hierarchical  vs.  flat) 

•  rotating  team  members 

Operational  Tasks 

•  team  decision  making,  COA 
selection 

•  develop  shared  understanding 

•  intelligence  analysis 
(team  data  processing) 


The  model  is  a  synthesis  of  the  literature  in  team  collaboration,  human  information 
processing  and  team  communication  together  with  the  results  obtained  during  the  annual 
workshop  on  Collaboration  and  Knowledge  Management  (Letsky,  2003).  During  the 
Collaboration  and  Knowledge  Management  (CKM)  workshop  twelve  initial  conceptual 
models  of  team  collaboration  were  produced  each  providing  some  unique  information 
along  with  overlapping  information.  The  models  varied  in  their  approach  from 
information-processing,  team  recognition  primed  decision  making,  transactive  memory, 
discovery  and  innovation,  and  hybrids  including  multi-stage  and  process  models.  The 
collaboration  stages  within  the  model  were  selected  by  determining  the  minimum  number 
of  unique  stages  identified  in  the  team  collaboration  literature  together  with  the  twelve 
models.  These  stages  also  had  to  be  supported  by  some  empirical  research.  The  cognitive 
processes  required  during  team  collaboration  are  based  on  an  information-processing 
approach,  which  also  includes  knowledge  building  for  supporting  the  information 
processing  tasks.  In  this  model  knowledge  is  built  through  team  communication  with  the 
initial  communication  mechanisms  derived  from  previous  research  (Stahl,  2000;  Cooke, 
2003;  Warner,  Vanderwalker  and  Verma,  2003). 

Model  Components 

Inputs  to  Model.  These  inputs  represent  general  information  that  is  required  prior  to 
team  collaboration.  This  information  includes  such  items  as:  (1)  a  description  of  the 
problem  task  to  be  solved,  (2)  team  member  expertise,  (3)  organizational  structure,  (4) 


Collaborative  Situation 

Parameters: 

•  time  pressure 

•  information/knowledge 
uncertainty 

•  dynamic  information 

•  large  amount  of  knowledge 
(cognitive  overload) 

•  human-agent  interface 
complexify 


Meta-Cognitive: 


•  individual  understanding  of 
problem  conditions 

•  individual  mental  model  developmei 
of  situational  significance 

Information  Processing! 

•  goal  development 

•  team  mental  model  of  problem 
rt  •  team  plan  to  solve  problem 

•  track  team's  mental  model  changes 

•  understanding  remaining  items  to 
resolve 

•  compare  problem  solution 
against  goals 

•  problem  identification 

•  understanding  problem  task 

•  establish  team  communication 
aid  trust 

•  establish  data  filtering  methods 

•  establish  meaning  transfer  conventic 

•  goal  definition 

•  iterative  information  collection 
&  analysis 

•  develop,  rationalize,  &  visualize 
solution  alternatives 

ns 

•  team  negotiation  of  solution 
alternatives 

•  analyze,  revise  output 

•Knowledge  Building: 

•  problem  definition 

•  individual  task  knowledge 

•  individual  team  knowledge 

•  team  mental  model  of  team 

•  team  task  knowledge 

•  domain  expertise 

•  shared  understanding 

•  collaborative  knowledge 

•  collaborative  knowledge 

•  shared  understanding 

•  goal  requirements 

•  exit  criteria 

Communication  Mechanism  for  Information  Processing  and  Knowledge  Building  (applies  to  all  stages): 

•  presenting  individual  information  •  dismissing  individual  information  •  dismissing  team  generated  information 

•  negotiating  perspectives  •  providing  rationale  fa  individual  solutions 

•  discussion  of  possible  solutions  •  agreement 


•  questioning 


Figure  2:  Structural  Model  of  Team  Collaboration. 
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team  members  roles  and  responsibilities  and  (5)  projected  events/future  information.  This 
representative  domain  information  is  provided  to  the  team  during  team  formation. 

Collaboration  Stages  and  Cognitive  Processes.  The  model  has  four  unique  but 
interdependent  stages  of  team  collaboration.  The  stages  are:  Team  Knowledge  Base 
Construction,  Collaborative  Team  Problem  Solving,  Team  Consensus  and  Product 
Evaluation  &  Revision.  There  is  also  a  feedback  loop  for  revising  team  solutions.  The 
stages  are  not  strictly  sequential  as  it  may  appear  in  Figure  2.  The  team  will  start  in  the 
Team  Formation  stage  and  proceed  onto  Collaborative  Team  Problem  Solving,  Team 
Consensus  and  Product  Evaluation  &  Revision.  However,  team  communication  is  very 
dynamic  and  moves  across  the  stages  throughout  the  collaboration  process.  The  cognitive 
processes  are  represented  at  four  levels:  Meta-cognition,  which  guides  the  overall 
problem  solving  process,  the  Information  Processing  Tasks  required  by  the  team  to 
complete  each  collaboration  stage,  the  Knowledge  Required  to  support  the  information 
processing  tasks  and  the  Communication  Mechanisms  for  Knowledge  Building  and 
Information  Processing.  For  each  collaboration  stage  the  respective  supporting  cognitive 
processes  will  be  described. 

Team  Knowledge  Base  Construction  is  the  first  step  in  team  collaboration  and  begins  by 
identifying  the  relevant  domain  information  required,  selecting  the  required  team 
members  and  setting  up  the  communication  environment  necessary  to  address  the 
problem.  The  type  of  teams  representative  in  this  model  have  characteristics  such  as 
being  asynchronous,  distributed,  culturally  diverse,  having  heterogeneous  knowledge, 
and  rotating  team  members  (see  Problem  Area  Characteristics  for  complete  list). 

Meta-cognitive  Process:  The  meta-cognitive  component  in  Team  Knowledge 
Base  Construction  is  the  Team ’s  understanding  of  the  elements,  relations  and  conditions 
that  compose  the  initial  state  of  the  problem.  Also  part  of  this  meta-cognitive  process  is 
for  each  team  member  to  construct  an  individual  mental  model  for  use  in  developing 
team  understanding. 

Information  Processing  Tasks:  The  team  has  several  information  processing 
tasks  to  perform  during  the  team  formation  stage.  The  team  has  to  collaborate  to  identify 
the  problem  to  be  solved,  to  understand  the  problem  task,  and  establish  team 
communication  and  team  trust.  In  identifying  and  understanding  the  problem  task  the 
team  must  recognize  that  a  problem  exist  before  they  can  solve  it.  Critical  elements  of  the 
problem  need  to  be  encoded  in  short-term  memory  while  information  relevant  to  these 
elements  are  retrieved  from  long-term  memory.  The  team  can  then  collaborate  on  what  is 
known,  unknown  and  what  is  being  asked  in  the  problem  situation. 

Knowledge  Required:  Three  types  of  knowledge  need  to  be  built  and  used  by  the 
team  members  in  support  of  completing  the  information  tasks.  Each  team  member  needs 
to  develop  his  or  her  individual  knowledge  of  the  task.  The  team,  as  a  whole,  needs  to 
develop  the  team ’s  knowledge  of  the  task.  Individual  and  team  task  knowledge  does  not 
have  to  be  exactly  the  same.  The  third  knowledge  type  is  the  team ’s  knowledge  of  the 
problem  to  be  solved. 
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Communication  Mechanisms  for  Knowledge  Building  and  Information 
Processing:  In  order  for  the  team  to  build  the  required  knowledge  needed  to  support  the 
information  processing  tasks  the  team  will  use  verbal  and  /or  written  communication 
between  team  members.  The  team  when  collaborating  on  the  information  processing 
tasks  uses  these  same  mechanisms.  The  specific  communication  mechanisms  are: 
presenting  individual  information,  discussing  individual  information,  discussing  team 
generated  information,  negotiating  perspectives,  providing  rationale  for  individual 
solutions,  agreement,  disagreement,  questioning  and  discussion  of  possible  solutions. 

Collaborative  Team  Problem  Solving  is  the  next  stage  and  begins  following  completion 
of  the  Team  Knowledge  Base  Construction  information  processing  tasks.  The 
collaborative  Team  Problem  Solving  stage  is  where  the  majority  of  collaboration  occurs 
among  team  members.  The  team’s  main  objective  in  this  stage  is  to  develop  viable 
solutions  to  the  problem. 

Meta-Cognitive  Process:  The  meta-cognitive  processes  involved  in  this  stage 
includes,  overall  goal  development,  developing  a  team  mental  representation  of  the 
problem  and  planning  how  the  team  will  solve  the  problem.  The  team  mental 
representation  can  change  during  the  course  of  solving  the  problem.  Changes  can  occur 
as  the  team  gains  more  complete  understanding  of  the  problem  elements,  goals  or 
overlooked  information.  According  to  Davidson  and  Sternberg  (1984,  1986),  new  mental 
representations  are  constructed  through  three  related  mental  processes:  selective 
encoding,  selective  combination,  and  selective  comparison.  Selective  encoding 
restructures  the  team  mental  representation  so  that  information  that  was  originally  viewed 
as  being  irrelevant  is  now  seen  as  relevant  for  problem  solution.  Also,  information  that 
was  originally  seen  as  relevant  may  now  be  viewed  as  irrelevant  and  eliminated  from  the 
team’s  mental  representation.  Selective  combination  involves  putting  together  elements  of 
a  problem  task  in  a  way  that  previously  was  not  obvious  to  the  team.  This  new  way  of 
combining  the  problem  elements  results  in  a  change  in  the  mental  representation. 
Selective  comparison  involves  discovering  relationships  that  are  not  obvious  between 
new  information  and  information  acquired  in  the  past.  The  realization  of  a  relationship 
between  new  and  old  information  results  in  a  change  to  the  mental  representation.  After 
the  problem  has  been  identified  and  mentally  represented,  the  team  must  decide  which 
steps  to  use  in  solving  the  problem.  Planning  often  involves  dividing  the  problem  into 
sub-problems  and  then  devising  a  sequence  for  how  the  sub-problems  should  be 
completed  (Flower  &  Hayes,  1981).  There  are  three  characteristics  of  planning  (Pea  & 
Hawkins,  1987).  First,  the  team  is  more  likely  to  engage  in  planning  when  the  problem 
situation  is  novel  and  complex  because  there  are  no  known  strategies  to  follow.  Second, 
planning  tends  to  be  abstract  rather  than  concrete  because  the  team  revises  their  plan 
based  on  how  well  it  is  working  and  available  opportunities  for  modifications.  Third, 
plans  have  both  costs  and  benefits,  which  involves  time  and  cognitive  resources  but 
overall  improve  problem  solving  efficiency.  In  team  problem  solving  there  are  four 
heuristics  that  are  often  used  (Greeno  &  Simon,  1988).  One  is  the  means-ends  analysis, 
which  tries  to  decrease  the  distance  between  the  team’s  current  position  in  the  problem 
space  and  where  the  team  wants  to  go  in  that  space.  A  second  heuristic  is  working 
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forward,  which  involves  starting  at  the  initial  problem  state  working  toward  the  desired 
state.  The  third  heuristic  is  working  backward,  which  involves  starting  at  the  desired  state 
and  trying  to  work  back  to  the  initial  state.  The  fourth  heuristic  is  generate  and  test, 
which  involves  generating  alternative  courses  of  action  and  evaluating  whether  each 
course  will  work. 

Information  Processing  Tasks:  In  order  to  develop  problem  solutions  the  team 
performs  several  information  processing  tasks.  These  tasks  include  definition  of  the  team 
goal,  iterative  information  collection  and  analysis,  and  development,  rationalization  and 
visualization  of  solution  alternatives. 

Knowledge  Required:  Five  types  of  knowledge  are  required  in  support  of 
completing  the  information  tasks.  Each  team  member  needs  to  continue  to  develop  his  or 
her  individual  knowledge  of  the  task.  The  team,  as  a  whole,  needs  to  continue  to  develop 
the  team  \s  knowledge  of  the  task.  The  team  in  addition  needs  to  develop  shared 
understanding  (i.e.  joint  agreement  of  the  facts  and  recognition  of  team  member 
perspectives  but  not  joint  acceptance).  Collaborative  knowledge  (i.e.  team  negotiation  of 
perspectives  resulting  in  a  deeper  understanding  and  team  agreement  of  the  facts)  also 
needs  to  be  developed  by  the  team.  The  last  knowledge  type  is  domain  expertise,  which  is 
not  developed  by  the  team  during  collaboration  but  is  required  by  team  members  to 
successfully  perform  the  information  processing  tasks. 

Communication  Mechanisms  for  Knowledge  Building  and  Information 
Processing:  During  the  Collaborative  Team  Problem  Solving  stage  the  team  will  use 
some  or  all  of  the  communications  mechanisms  discussed  earlier  to  perform  the 
information  processing  tasks  and  knowledge  building. 


Team  Consensus  is  the  next  collaboration  stage  and  begins  when  the  team  has  several 
viable  solution  alternatives  to  the  problem.  The  main  objective  of  team  consensus  is  to 
achieve  team  agreement  of  the  common  output. 

Meta-cognitive  Process:  The  meta-cognitive  processes  involved  in  this  stage 
includes,  the  team  keeping  track  of  what  they  have  already  done,  what  they  are  currently 
doing  and  what  needs  to  be  done.  In  other  words,  the  team  needs  to  track  changes  to  the 
team’s  mental  model. 

Information  Processing  Tasks:  In  order  for  the  team  to  achieve  agreement  of  the 
common  output,  the  team’s  mental  model  of  the  solution  must  converge.  The 
information-processing  task  under  this  stage  is  team  negotiation  of  the  solution 
alternatives. 

Knowledge  Required:  To  achieve  convergence  of  the  team’s  mental  model  and 
successful  team  negotiation  of  solution  alternatives  two  types  of  knowledge  are  required: 
(1)  shared  understanding  (i.e.  joint  agreement  of  the  facts  and  recognition  of  team 
member  perspectives  but  not  joint  acceptance)  and  (2)  collaborative  knowledge  (i.e.  team 
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negotiation  of  perspectives  resulting  in  a  deeper  understanding  and  team  agreement  of  the 
facts). 


Communication  Mechanisms  for  Knowledge  Building  and  Information 
Processing:  During  the  Team  Consensus  stage  the  team  will  use  some  or  all  of  the 
communications  mechanisms  discussed  earlier  to  perform  the  information  processing 
tasks  and  knowledge  building. 

Outcome  Evaluation  and  Revision  is  the  final  stage  of  collaboration.  The  main  objective 
of  this  stage  is  to  analyze,  test  and  validate  the  agreed  upon  team  solution  against  the  goal 
requirement(s)  and  exit  criteria.  Included  in  this  stage  is  an  iteration  loop  for  deriving 
other  solutions  for  the  problem  if  necessary. 

Meta-cognitive  Process:  The  meta-cognitive  processes  involved  in  this  stage 
include  comparing  the  problem  solution  against  the  goal(s). 

Information  Processing  Tasks:  The  information-processing  task  under  this  stage 
is  for  the  team  to  analyze  and  revise  the  problem  solution,  if  necessary. 

Knowledge  Required:  In  order  for  the  team  to  perform  the  information¬ 
processing  task,  two  types  of  knowledge  are  required:  (1)  goal  requirements  and  (2)  exit 
criteria  for  viable  solutions. 

Communication  Mechanisms  for  Knowledge  Building  and  Information 
Processing:  During  the  Outcome  Evaluation  &  Revision  stage  the  team  will  use  some  or 
all  of  the  communications  mechanisms  discussed  earlier  to  perform  the  information 
processing  tasks  and  knowledge  building. 

Model  Outputs.  The  output  of  the  model  reflects  the  type  of  product  from  the  team 
collaboration  process.  The  product  type  will  vary  depending  on  the  problem  domain 
addressed  by  the  team.  This  structural  model  of  team  collaboration  focuses  on  the 
following  product  types:  selected  course  of  action(s),  recommendations,  situation 
assessment,  risk  assessment,  product  or  tool,  opinion,  guidelines. 

The  proposed  structural  model  of  team  collaboration  is  intended  to  be  a  starting  point  for 
understanding  the  cognitive  mechanisms  of  team  collaboration  given  the  collaborative 
problem  area  characteristics  presented  in  Figure  2.  This  model  also  offers  a  model-based 
approach  to  experimentation  of  team  collaboration,  which  aids  in  focusing  specific 
research  experiments  while  enhancing  the  model  through  new  research  findings.  The 
following  experiment  was  designed  to  provide  empirical  data  on  the  validity  of  the 
collaboration  stages  and  cognitive  processes  of  the  structural  model.  The  experiment  had 
two  independent  variables.  Collaboration  Mode  (i.e.  face-to-face  and  asynchronous, 
distributed)  and  Knowledge  Distribution  (i.e.  Homogeneous  and  Heterogeneous).  Results 
from  the  experiment  were  used  to  update  the  structural  model  of  team  collaboration. 
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Method 


Participants 

Eighty-four  undergraduate  students  from  a  community  college  in  Southern  Maryland 
served  as  participants  in  this  study.  Participants  included  twenty-three  men  and  sixty-one 
women,  ranging  in  age  from  16  to  58  years  of  age  (mean  =  22.92  years,  standard 
deviation  =  8.76).  Participants  were  paid  $15  per  hour  for  a  task  that  could  last  up  to  four 
hours  and  received  extra  credit  from  their  professor.  Students  were  randomly  assigned 
either  to  the  face-to-face,  homogeneous  teams  (n=seven  teams),  face-to-face 
heterogeneous  teams  (n=  seven  teams),  asynchronous,  distributed  homogeneous  teams 
(n=seven  teams)  or  asynchronous,  distributed  heterogeneous  teams  (n=seven  teams). 
Each  team  consisted  of  three  students. 


Apparatus 

Both  the  face-to-face  and  asynchronous,  distributed  teams  solved  the  collaborative 
problem  solving  task  in  the  Naval  Air  Systems  Command  Cockpit  Automation  and 
Research  Laboratory.  Figure  3  and  4  illustrate  the  face-to-face  and  asynchronous, 
distributed  environments  respectively. 


Figure  3:  Face-to-Face  Environment  Figure  4:  Asynchronous,  Distributed 

Environment 


Experimental  Task 

The  experimental  task  required  participants  to  work  in  teams  of  three  to  solve  a  murder 
mystery.  After  reading  The  Case  of  the  Fallen  Businessman  (Stasser,  1995),  the  teams 
were  required  to  reach  consensus,  identifying  which  of  the  three  possible  suspects  they 
believed  committed  the  murder.  During  collaborative  problem  solving  the  face-to-face 
teams  solved  the  mystery  through  direct  speech  discussions  around  a  conference  table 
while  the  asynchronous,  distributed  teams  used  a  web-based  text  forum  to  communicate 
with  other  team  members.  To  simulate  an  asynchronous  environment,  a  randomized 
delay  between  1  and  60  seconds  was  used  to  delay  the  message  from  the  time  it  was 
posted  by  a  team  member  to  the  time  it  was  displayed  on  the  forum.  Team  members 
assigned  to  the  homogenous  groups  received  identical  information  about  the  murder 
mystery.  Participants  assigned  to  the  heterogeneous  groups  received  common  information 
as  well  as  information  unique  to  each  team  member.  Heterogeneous  team  members  were 
not  informed  of  the  script  variations. 
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The  web-based  forum  was  hosted  on  a  PC  server  and  presented  to  asynchronous, 
distributed  team  members  on  their  IBM  laptop.  All  asynchronous,  distributed  team 
members  were  in  separate  office  modules  within  the  laboratory.  All  face-to-face 
communication  was  audio  and  video  recorded  and  timed  tagged  for  later  transcription 
while  all  the  web-based  forum  text  was  recorded,  time  tagged  and  saved  real-time. 

Procedure 

After  being  assigned  to  one  of  the  four  experimental  groups  (face  to  face  homogeneous, 
face  to  face  heterogeneous,  asynchronous  distributed  homogeneous  or  asynchronous 
distributed  heterogeneous),  participants  reviewed  and  completed  informed  consent  forms. 
All  teams  were  then  given  a  brief  introduction  to  the  murder  mystery  task.  In  addition, 
asynchronous  distributed  teams  were  instructed  on  the  use  of  the  web-based  forum. 
Following  the  instruction  period,  they  were  outfitted  with  hearing  protection,  seated  at  an 
isolated  computer  station  and  given  a  five-minute  practice  session.  Following  the 
procedural  introduction,  team  members  were  given  twenty  minutes  to  independently  read 
The  Case  of  the  Fallen  Businessman  (Stasser,  1995).  The  script  given  to  them  was 
specific  to  their  assigned  knowledge  distribution  (homogeneous  or  heterogeneous). 
While  subjects  were  allowed  to  take  notes  when  reading,  interaction  and  discussion 
among  subjects  was  prohibited.  Participants  were  then  instructed  to  reach  a  group 
consensus  regarding  the  identity  of  the  murderer.  Previous  studies  (Cristian,1996; 
Goodman, et.al., 2001)  comparing  face  to  face  and  asynchronous,  distributed  collaborative 
teams  have  shown  that  face  to  face  teams  complete  collaborative  problems  faster  than 
asynchronous,  distributed  teams.  For  this  experiment  this  effect  was  to  be  minimized. 
Each  of  the  four  experimental  groups  had  a  maximum  time  to  complete  the  problem.  The 
maximum  collaboration  time  allowed  varied  with  each  experimental  group  based  on 
previous  pilot  studies.  Face  to  face  homogeneous  teams  were  allowed  15  minutes  to  reach 
a  consensus,  while  face  to  face  heterogeneous  groups  were  allowed  30  minutes. 
Likewise,  asynchronous  homogeneous  groups  were  allowed  60  minutes  for  consensus 
and  asynchronous  heterogeneous  groups  were  given  75  minutes.  Groups  that  reached 
consensus  before  the  allotted  time  expired  were  allowed  to  end  the  session. 

Dependent  Variables 

Five  dependent  variables  were  identified:(l)  accuracy  of  decision,  (2)  time  for  task 
completion,  (3)  time  spent  in  each  collaboration  stage  and  cognitive  process  state,  (4) 
frequency  of  utterances,  and  (5)  transition  probabilities. 

Accuracy  of  Decision:  The  collaborative  task  was  designed  to  provide  all  of  the 
information  necessary  to  correctly  identify  a  specific  suspect  as  the  murderer. 
Performance  was  assessed  through  a  comparison  of  the  percentage  of  correct  decisions  by 
collaboration  mode  and  knowledge  distribution. 

Time  for  Task  Completion:  In  an  effort  to  compensate  for  variations  in  task  difficulty, 
the  maximum  collaboration  time  allowed  varied  with  each  experimental  group.  The 
adjusted  mean  percent  of  time  for  task  completion  was  assessed  in  relation  to 
collaboration  mode  and  knowledge  distribution.  This  was  accomplished  by  comparing 
the  time  allotted  to  the  actual  time  used  for  each  team. 
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Time  Spent  in  each  Collaboration  Stage  and  Cognitive  Process  State:  The  time  each 
team  spent  in  each  collaboration  stage  and  process  state  was  analyzed  with  respect  to  the 
total  time  needed  for  task  completion. 

Frequency  of  Utterances:  An  analysis  of  the  number  of  utterances  within  each 
collaboration  stage  and  process  state  was  performed. 

Transition  Probabilities:  Transition  probabilities  indicate  the  likelihood  that  one 
collaboration  stage  will  follow  another.  Transition  probabilities  for  cognitive  process 
states  were  also  analyzed. 

Results 

A  Chi  Square  test  of  independence  was  used  to  analyze  accuracy  of  team  decisions  across 
collaboration  mode  and  knowledge  distribution  conditions.  A  2x2  randomized  factorial 
analysis  of  variance  was  used  to  analyze  time  for  task  completion  across  collaboration 
mode  and  knowledge  distribution  along  with  time  spent  and  frequency  of  utterances  in 
collaboration  stages.  Two  trained  raters  using  verbal  protocol  analysis  with  a  pre-defined 
coding  scheme  to  categorize  each  utterance  with  respect  to  collaboration  stage  and 
process  state  performed  the  initial  analysis  of  the  collaboration  stages  and  cognitive 
processes.  The  specific  coding  categories  and  their  definitions  are  described  in  appendix 
A.  Multiple  analysis  of  variance  (MANOVA)  analyses  were  used  to  analyze  time  spent 
along  with  frequency  of  utterances  in  each  cognitive  process  state.  For  all  statistical 
analyses  an  alpha  level  of  .05  was  used  for  statistical  significance. 

There  was  no  significant  effect  of  decision  accuracy  across  collaboration  mode  and 
knowledge  distribution  conditions,  Yates  corrected  Chi  Square  (1)  =  .27,  p=.6056.  There 
was  no  significant  effect  for  mean  percent  time  for  task  completion  across  collaboration 
mode,  F(l,82)  =  0.6406,  p  =  0.431353  or  knowledge  distribution,  F(l,82)  =  0.1491,  p  = 
0.702782,  or  the  interaction,  F(3,80)  =  0.0006,  p  =  0.933647.  Figure  5  illustrates  a 
significant  effect  of  the  mean  percent  time  spent  in  the  four  collaboration  stages  by  the 
four  different  team  types.  During  team  knowledge  construction  the  face-to-face 
heterogeneous  teams  spent  longer  in  this  stage  than  the  asynchronous,  distributed 
heterogeneous  teams.  During  collaborative  team  problem  solving  the  asynchronous, 
distributed  teams  spent  more  time  problem  solving  than  the  face-to-face  teams.  The  face- 
to-face  heterogeneous  teams  had  a  significantly  higher  frequency  of  utterances,  F(3,80)  = 
5.64772,  p  =  0.025802,  regarding  team  knowledge  construction  than  the  other  three  team 
types.  Figure  6  illustrates  the  two  cognitive  states  that  were  significantly  different  during 
team  knowledge  construction  between  the  face-to-face  and  asynchronous,  distributed 
teams.  The  asynchronous,  distributed  teams  spent  significantly  more  time  understanding 
the  problem  compared  to  face-to-face  teams,  while  the  face-to-face  teams  spent  more 
time  building  team  knowledge  compared  to  asynchronous,  distributed  teams.  Figure  7 
illustrates  that  during  team  problem  solving,  face-to-face  teams  spent  significantly  more 
time  using  conventions  to  transfer  meaning  than  asynchronous,  distributed  teams  while 
the  asynchronous,  distributed  teams  spent  more  time  developing  solution  alternatives 
than  face-to-face  teams.  There  was  no  significant  difference  between  the  four-team  types 
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Collaboration  Mode  (CM)  vs.  Knowledge  Distribution  (KD) 


Team  Knowledge  Base  Construction: 

CM:  F  =5.51323,  p  =0.027449 
KD:  F  =  1.70682,  p  =  0.203780 
CMKD:  F  =6.02627,  p  =  0.021724 
N  =  84 


Cq llabQrativg  Team  PrQblgm  SQlving: 

CM:  F  =6.944,  p  =  0.014504 
KD:  F  =2.010,  p  =  0.169097 
CM*KD:  F  =  1.876,  p  =  0.183471 
N  =  84 


Tgam  CQn$gn$u$: 

CM:  F  =0.51854,  p  =0.478422 
KD:  F  =3.43855,  p  =0.07602 
CM*KD:  F  =0.82967,  p  =  0.371425 
N  =  84 


Collaboration  Mode /Knowledge  Distribution: 

□  Face  to  Face  Homogeneous  MFace  to  Face  Heterogeneous 

□  Asynchronous,  Distributed  Homogeneous  ■  Asynchronous ,  Distributed  Heterogeneous 
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*Ad justed:  Each  team's  time  per  stage  was 

Collaboration  Stages  calculated  by  dividing  the  utterance  time  for  each 
stage  by  the  total  utterance  time  for  the  team. 


Figure  5:  Mean  Percent  Time  by  Collaboration  Stage. 


Mean  %  Time  By  Cognitive  Process  States: 
Team  Knowledge  Construction  (TK) 

Wilks  larrtoda  =  0.402182,  F  =  3.158681,  p  =  0.022010 


0) 

.§  * 


c  +- 

(3  > 
Q  > 


80 

70 

60 

50 

40 

30 

20 

10 

0 


P:  P:  P:  KB:  KB:  KB: 


Understanding 

Goal 

Establishing 

Data  Filtering 

Transfer 

Problem 

the  Problem 

Definition 

Team  Trust 

(Pdf) 

Conventions 

Definition 

(Pup) 

**  =  significantly  different 

(Pgd) 

(Pet) 

(Ptck) 

Process  States 

(KBpd) 

Individual  Team 
Knowledge  Knowledge 
(KBik)  (KBtk) 


Figure  6:  Mean  Percent  Time  in  Team  Knowledge  Construction. 
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Mean  %Time  By  Gocfiitive  Process  States: 
Collaborative  Team  Problem  Solving  (TPS) 

Wlkslarrtda= 0.00033  F= 9605.498  p=0.000000 


<D 

12 


if 

<D  fe 


MC: 

F: 

P: 

P: 

KB:  KB: 

KB: 

Team  Plan 

Collection 

&  Transfer 

Solution 

Domain  Expertise  Shared 

Collaborative 

(MCtp) 

Anakysis 

Conventions 

Alternatives 

(Pde)  Understanding 

Knowledge 

(Pica) 

(IPtcp) 

(Psa) 

(KBsu) 

(KBck) 

**  =  significantly  different 

Figure  7:  Mean  Percent  Time  in  Collaborative  Team  Problem  Solving. 


in  the  cognitive  processes  used  during  team  consensus,  Wilks  lambda  =  0.815855,  F  = 
1.65519,  p  =  0.205648.  The  transition  probabilities  for  the  collaboration  stages  and 
cognitive  processes  are  presented  in  Figures  8  and  9  respectively. 


Transition  Probabilities: 
Collaboration  Stages 


Collaborative  Team 
Problem  Solving 
(TPS) 

Probability  of  Occurrence: 
F2F:  54%  AD:  76% 


Team  Knowledge 
Base  Construction 
(TK) 

Probability  of  Occurrence: 
F2F:  39%  AD:  12% 


Outcome, 
Evaluation  &  Revision 
(OER) 

Probability  of  Occurrence: 

F2F:  0%  AD:  0% 
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Figure  9:  Transition  Probabilities  for  Cognitive  Processes 


Discussion 

Both  face-to-face  and  asynchronous,  distributed  teams  demonstrated  behavior  that 
supports  the  existence  of  Team  Knowledge  Construction,  Collaborative  Problem  Solving 
and  Team  Consensus  stages  during  collaborative  problem  solving.  The  empirical  data 
showed  that  the  stages  are  task  dependent  as  the  Outcome  Evaluation  &  Revision  stage 
was  not  used.  The  significant  cognitive  process  states  were  Understanding  the  Problem, 
Individual  Knowledge  and  Team  Knowledge  development,  Conventions  for  Transferring 
Meaning,  and  developing  Solution  Alternatives.  Knowledge  Distribution  (homogeneous  / 
heterogeneous  information)  significantly  influenced  how  team  knowledge  was 
developed.  Asynchronous,  distributed  teams  spent  more  time  in  Collaborative  Problem 
Solving  stage  than  face-to-face  teams.  It  appears  to  be  more  difficult  to  solve 
collaborative  problems  with  these  types  of  teams  even  though  collaboration  environment 
is  inherently  more  structured.  Need  to  determine  why  these  teams  spend  more  time  so 
collaborative  problem  solving  can  be  facilitated.  Face-to-face  teams  used  conventions  to 
transfer  meaning  (e.g.  yellow  stickers,  maps)  whereas  asynchronous,  distributed  teams 
did  not  use  conventions.  Need  ways  to  easily  create  conventions  with  asynchronous, 
distributed  teams  (another  area  for  agent  support).  Face-to-face  teams  demonstrated 
mostly  a  linear  path  between  team  knowledge  construction,  team  problem  solving  and 
team  consensus  whereas  asynchronous,  distributed  teams  showed  a  non-linear  path  with 
feedback  loops  (area  for  agent  support).  In  summary,  the  empirical  data  from  this  study 


16 


has  shown  major  differences  between  face-to-face  and  asynchronous,  distributed  teams  in 
both  the  stages  and  the  cognitive  processes  used  during  team  collaboration.  Based  on 
these  differences,  capabilities  need  to  be  incorporated  within  new  collaboration  tools  that 
facilitate  asynchronous,  distributed  team  collaboration. 
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Appendix  A:  Coding  Categories  and  Definitions  for  Collaboration  Stages  and 
Cognitive  Processes 

Collaboration  Stages  Categories  &  Definitions 

•  Team  Knowledge  Base  Construction  (TK):  “Team  members  reading,  clarifying 
information  and  understanding  the  problem.” 

•  Defining  the  problem  (realizing  what  they  have  to  accomplish  — 

“We  need  to  come  up  with  a  solution  in  30  minutes.”) 

•  Clarifying  the  facts  —  “What  did  they  say  about  Joe’s  hearing? 

•  Collaborative  Team  Problem  Solving  (TPS):  “Team  members  working  together  to 
develop  solution  options  to  the  problem.” 

•  Analyzing  the  facts  to  come  up  with  a  solution  —  “I  think  Joe  did  it  because  he  tried 
to  blame  everything  on  Tim.” 

•  Team  Consensus  (TC):  “Team  negotiation  of  solution  option  and  final  agreement 
by  all  team  members  on  a  particular  option.” 

*  Agreeing  on  the  final  solution  —  “Do  we  all  agree  that  Joe  did  it?” 

•  Outcome  Evaluation  &  Revision  (OER):  “Team  evaluation  of  selected  solution 
option  against  problem  solving  goal.  Team  revises  solution  option  if  option  does  not 
meet  goal.” 

*  Choosing  to  accept  the  final  decision  or  revise  it - “I  know  we  all  agreed  on  Tim, 

but  I  think  we  better  take  a  closer  look  at  Joe.” 

•  Additional  Stages  (MISC):  “Other  unique  team  behavior  not  described  in  the 
above  categories.” 

*  Need  to  describe  unique  behavior  and  label  stage. 


Cognitive  Process  States:  Categories  &  Definitions 

Team  Knowledge  Base  Construction:  (TK)  “Team  members  reading  clarifying  and 
understanding  the  problem  to  be  solved.” 

IPup:  Information  Processing  (understanding  problem)  =  stating  the  facts  without 
applying  that  knowledge  to  any  possible  solution. 

“Tim’s  muffler  was  loud” 

IPgd:  Information  Processing  (goal  definition)  =  defining  the  team  goal. 

“Our  goal  is  to  determine  who  killed  Mr.  Gill” 

IPct:  Information  Processing  (communication  &  trust)  =  establishing  team  trust. 

“I  understand  why  you  thought  it  was  Tim.” 

IPdfm:  Information  Processing  (data  filtering  methods)  =  team  uses  methods  to  sort  data, 
“using  yellow  note  pads  to  sort  data  into  categories” 
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IPtck:  Information  Processing  (transfer  conventions)  =  team  uses  methods  to  transfer 
meaning  to  other  team  members  during  knowledge  construction. 

“team  uses  map  of  Mr.  Gill’s  house  to  understand  distances  between  objects. 
KBpd:  Knowledge  Building  (problem  definition)  =  defining  the  problem  (realizing  what 
they  have  to  accomplish) 

“We  need  to  come  up  with  a  solution  in  30  minutes” 

KBik:  Knowledge  Building  (indiv.  knowledge)  =  individual  clarifying  the  facts;  asking 
for  clarification. 

“What  did  they  say  about  Joe’s  hearing” 

KBtk:  Knowledge  Building  (team  knowledge)  =  team  clarifying  facts 
“Billy  left  the  coffee  shop  at  7:00am” 

“No  that  was  Eddie  that  left  the  coffee  shop  at  7:00am 
“Yes  that  was  Eddie” 

Collaborative  Team  Problem  Solving:  (TPS)  “Team  members  working  together  to 
develop  solution  options  to  the  problem.” 

MCtp:  Meta-cognitive  (team  plan)  =  establishing  a  plan  of  approach. 

“ADHES  2,  why  don’t  you  say  who  you  think  it  is,  then  I’ll  say  who  I  think  it  is.” 
IPica:  Information  Processing  (information  collection  &  analysis)  =  collecting  and 
analyzing  the  facts  to  come  up  with  a  solution  but  no  specific  solution  mentioned. 

“The  killer  seems  to  blame  everything  on  Tim.” 

IPtcp:  Information  Processing  (transfer  conventions)  =  team  uses  methods  to  transfer 
meaning  to  other  team  members  during  team  problem  solving. 

“  Using  maps  or  yellow  stickers  to  transfer  meaning  to  team  members” 

IPsa:  Information  Processing  (solution  alternatives)  =  developing,  rationalizing  and 
discussing  solution  alternatives. 

“I  think  Eddie  did  it?” 

KBde:  Knowledge  Building  (domain  expertise)  =  team  members  state  their  own  domain 
knowledge  and  apply  that  knowledge  to  the  problem. 

“Yes  I  am  a  Maryland  State  Trooper  and  they  is  no  concrete  evidence  for  Billy 
killing  Mr.  Gill.” 

KBsu:  Knowledge  Building  (shared  understanding)  =  using  facts  to  justify  a  solution. 

“I  think  Eddie  did  it  because  he  was  hard  of  hearing  and  had  his  fingerprints  on 
the  crowbar.” 

KBck:  Knowledge  Building  (collaborative  knowledge)  =  Convincing  others  of  a  specific 
thought  without  absolute  consensus  of  the  final  solution. 

“You’re  right.  I  didn’t  think  about  that.” 

Team  Consensus:  (TC)  “Team  negotiation  of  solution  option  and  final  agreement 
by  all  team  members  on  a  particular  option.” 

MCitr:  Meta-cognitive  (items  to  resolve)  =  team  stating  remaining  items  to  resolve. 

“We  need  to  compare  Mickey’s  and  Eddie’s  location  times  with  Billy’s” 

IPtn:  Information  Processing  (team  negotiation)  =  team  negotiation  of  solution 
alternatives  ending  in  a  final  solution. 

“  I  believe  it  is  Eddie  because  he  was  hard  of  hearing” 

“  I  concur  which  also  means  Eddie  was  lying  about  where  is  was” 
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“I  also  think  it  was  Eddie  because  of  his  fingerprints  on  the  crowbar. 

“We  all  in  consensus,  it  was  Eddie” 

KBckc:  Knowledge  Building  (collaborative  knowledge)  =  convincing  others  of  a 
specific  thought  without  absolute  consensus  of  the  final  solution. 

“You’re  right.  I  didn’t  think  about  that.” 

KBsuc:  Knowledge  Building  (shared  understanding)  =  using  facts  to  justify  a  solution. 

“It  couldn’t  have  been  Tim  because  his  muffler  was  loud  and  the  car  at  the  coffee 
shop  had  a  quiet  muffler.” 

Outcome  Evaluation  &  Revision:  (PER)  “Team  evaluation  of  selected  solution 
option  against  problem  solving  goal.  Team  revises  solution  option  if  option  does  not 
meet  goal.” 

MCps:  Meta-cognitive  (problem  solution)  =  compare  problem  solution  against  goal(s). 

“Are  we  done  or  do  we  want  to  talk  about  it  some  more?” 

IPar:  Information  Processing  (analyze,  revise  output)  =  analyze  final  solution  option  and 
revise  if  necessary. 

“I  know  we  all  agreed  on  Tim,  but  I  think  we’d  better  take  a  closer  look  at  Joe.” 
KBgo:  Knowledge  Building  (goal  obtainment)  =  understand  if  solution  option  fits 
specific  goal  criteria. 

“  the  murder  had  a  truck  with  a  loud  muffler,  was  at  Mr.  Gills  house  around 
10:30am,  and  pushed  Mr.  Gill  down  the  porch  steps”. 
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Appendix  B:  Glossary  of  Terms 

Collaboration:  the  process  of  shared  creation;  two  or  more  individuals  with  complementary  skills 
interacting  to  create  a  shared  understanding  that  none  had  previously  possessed  or  could  have  come 
to  on  their  own.  The  cognitive  aspects  of  joint  problem  solving  for  the  purpose  of  attaining 
knowledge  sufficient  to  complete  the  common  task. 

Shared  Understanding:  The  process  of  reaching  team  agreement  about  the  characteristics  and  meaning 
of  the  situation  at  hand  within  the  bounds  of  the  objectives  of  the  collaborative  activity  (decision  making, 
intelligence  analysis,  COA  selection,  etc.).  Not  all  team  members  will  have  equal  depth  and  breadth  in 
all  supporting  knowledge  but  each  has  sufficient  knowledge  to  complete  his  role  as  a  team  member. 

Asynchronous,  Distributed  Collaboration:  a  cohesive  group  of  individuals  working  at  different  locations 
and  at  different  times  to  solve  a  common  task. 

Mental  Model:  a  knowledge  structure  that  represents  information. 

task. 

meaning  as  they  engage  with  texts  by  organizing,  selecting,  and  connecting  information; 
making  inferences;  and  performing  acts  of  interpretation. 

Computational  Models:  Calculational  tool  that  implements  a  set  of  mathematical 
equations  designed  to  represent  a  conceptual  model. 

Conceptual  Models: 

(1)  Set  of  qualitative  assumptions  used  to  describe  a  system  (or  part 
thereof).  These  assumptions  may  cover  the  geometry  and 
dimensionality  of  the  system,  initial  and  boundary  conditions,  time 
dependence,  and  the  nature  of  the  relevant  physical,  chemical  and 
biological  processes  and  phenomena. 

(2)  Consists  of  a  set  of  assumptions  that  reduce  the  real  problem  and  the 
real  domain  to  simplified  versions  that  are  satisfactory  in  view  of  the 
modeling  objectives  and  the  associated  problem. 

Consensus:  Opinion  or  position  reached  by  the  group  as  a  whole. 

Data:  Factual  information  (as  measurements  or  statistics)  used  as  a  basis  for  reasoning, 
discussion,  or  calculation.  Data  on  its  own  has  no  meaning,  but  becomes  information 
when  it  is  interpreted. 


Decision-Making:  Form  of  problem  solving  in  which  one  tries  to  make  the  best  choice 
from  among  alternative  judgments  or  courses  of  action. 

Human- Agent  Interface:  A  connection  point  that  allows  for  the  interaction  between  a 
user  and  software,  which  carries  out  some  set  of  operations  on  behalf  of  a  user  or  another 
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program  with  some  degree  of  independence  or  autonomy,  and  in  so  doing,  employ  some 
knowledge  or  representation  of  the  user’s  goals  or  desires. 

Knowledge  Building: 

(1)  Process  through  which  we  increase  both  our  individual  and  our 
common  understanding. 

(2)  Theory  of  learning,  which  emphasizes  the  collaborative  construction  of 
knowledge  by  a  group  of  learners. 

Situational  Awareness:  Person’s  mental  model  of  the  current  state  of  a  dynamic 
environment;  the  perception  of  the  elements  in  the  environment  within  a  volume  of  time 
and  space,  the  comprehension  of  their  meaning  and  the  projection  of  their  status  in  the 
near  future. 

Transactive  Memory:  Consists  of  the  collection  of  individual  understandings  and  the 
team  mechanisms  to  exchange  information,  which  update  these  individual 
understandings. 
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Objective  of  Presentation 
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>  To  describe  the  unique  cognitive  processes  that  are  employed  to  optimize 
collaborative  team  decision  making  in  a  geographically  distributed  and  time 
delayed  situation 


>  To  describe  an  empirically-based  structural  model  of  team  collaboration  that 
illustrates  the  respective  cognitive  processes 
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Major  Factors  Impacting  Team  Collaboration 


NAV^AIR 


Collaborative  Challenges 

Increasing  problem  complexity  -  team  effort  needed 
IT/Communications  technology  widening  accessibility  of  contributors 
Problems  addressed  at  international  level  -  coalitions  required 
Defense  Transformation  to  agile  and  coalition  operations 
Information  overload  condition 


Operational  Tasks: 

*  Team  decision  making,  CO  A  selection 

*  Develop  shared  understanding 

:  Intell  analysis  (team  data  processing) 


Collaborative  Situation  Parameters: 


*  time  pressure 

*  information  /  knowledge  uncertainty 

*  dynamic  information 

*  large  amount  of  knowledge 
(cognitive  overload) 

*  human  -agent  interfaces 


*  asynchronous 

*  distributed 

*  culturally  diverse 

*  heterogeneous  knowledge 


Team  Types 


*  unique  roles 

*  command  structure 
( hierarchical  vs  flat) 

*  rotating  team  members 
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Impact  of  Theories  of  Cognitive  Psychology 

on 

Team  Collaboration  Model  Development 
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Theories  of  Human  Cognition 

>  Origin  of  Human  Thought  and  Knowledge  -  Descartes  (1641)  &  Kant  (1781) 

>  Behaviorism  -  Skinner  (1985) 

>  Information-Processing  Model  -Turing  (1936),  Weiner  (1948),  Shannon  (1949) 

Wickens  (1992) 

>  Human  Language  -  Chomsky  (1957),  Cooke  (2003) 

>  Developmental  Biology  -  Piaget  (1970) 

>  Computer  Computational  Model  -  Newell  &  Simon  (1956),  Anderson  (1993), 

Minsky  (1997) 

>  Physiological  Neural  Networks  -  Rumelhart  (1990),  Churchland  (1989) 

>  Meta-Cognition  -  Davidson,  Deuser  &  Sternberg  (1994) 

Impact  on  Model  Development 

>  Multi-Disciplinary  Approach  to  Cognition 

>  No  Unified  Theory  of  Human  Cognition 

>  Insufficient  Objective  Metrics  to  Measure  Human  Cognitive  Processes 
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STRUCTURAL  MODEL  OF  TEAM  COLLABORATION 


Problem  Area 
Characteristics 


Collaborative  Situation 


Parameters: 


•  time  pressure 

•  information/knowledge 
uncertainty 

•  dynamic  information 

•  large  amount  of  knowledge 
(cognitive  overload) 

•  human-agent  interface 
complexity 

Team  Types 


•  asynchronous 

•  distributed 

•  culturally  diverse 

•  heterogeneous  knowledge 

•  unique  roles 

•  command  structure 
(hierarchical  vs.  flat) 

•  rotating  team  members 

Operational  Tasks 


•  team  decision  making,  COA 
selection 

•  develop  shared  understanding 

•  intelligence  analysis 
(team  data  processing) 


A 

Meta-Cognitive: 


Team 
Knowledge 
Base 

Construction 


1 


Collaboration  Stases 

k: 


Collaborative 
Team  Problem 
Solving 


Team 
Consensus 


Collaboration 

Complete 


Outcome 
Evaluation 
and  Revision 


Yes 


/  /  Achieve' 

Goal 


No 


•  individual  understanding  of 
problem  conditions 

•  individual  mental  model  developmei 
of  situational  significance 

Info rmation  Processi ug; . 

•  goal  development 

•  team  mental  model  of  problem 

•  team  plan  to  solve  problem 

•  track  team's  mental  model  changes  ’ 

•  understanding  remaining  items  to 
resolve 

•  compare  problem  solution  j 
against  goals 

•  problem  identification 

•  understanding  problem  task 

•  establish  team  communication 
and  trust 

•  establish  data  filtering  methods 

•  establish  meaning  transfer  conventic 

•  goal  definition 

•  iterative  information  collection 
&  analysis 

•  develop,  rationalize,  &  visualize 
solution  alternatives 

jns 

•  team  negotiation  of  solution 
alternatives 

•  analyze,  revise  output 

•Knowledge  Building: 

•  problem  definition 

•  individual  task  knowledge 

•  individual  team  knowledge 

•  team  mental  model  of  team 

•  team  task  knowledge 

•  domain  expertise 

•  shared  understanding 

•  collaborative  knowledge 

•  collaborative  knowledge 

•  shared  understanding 

•  goal  requirements 

•  exit  criteria 

Communication  Mechanism  for  Information  Processing  and  Knowledge  Building  (applies  to  all  stages): 


►  presenting  individual  information 

*  disagreement 

*  questioning 


•  discussing  individual  information 
• negotiating  perspectives 

•  discussion  of  possible  solutions 


*  discussing  team  generated  information 

*  providing  rationale  for  individual  solutions 

*  agreement 
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Experiment  1  Collaborative  Problem  Solving  Tas 

Murder  Mystery 


faAV^AIR 


•  The  Case  of  the  Fallen  Businessman,  ( by  Dr.  Garold  Stasser,  Miami  University,  Oxford,  Ohio) 


Major  Characters 

Robert  Gill: 
Mary  Gill: 

Lt.  Mark  Moody: 
Sgt.  Cassini: 
**Eddie  Sullivan: 
**Billy  Prentice: 


The  victim 
The  victim’s  wife 

Detective  in  charge  of  the  investigation 
Police  officer  assisting  in  the  investigation 
Handyman  who  worked  for  the  Gills 
Yardman  who  worked  for  the  Gills 


**Mickey  Malone:  Owner  of  MM  Auto  Parts;  business  associate 


Sam  Nietzel: 
Dave  Daniels: 


Parts  manager  for  Gill  Lincoln/Mercury 
Owner  of  Dave’s  Quick  Stop  in  the  Eastwood 


**  The  ONLY  suspects  under  consideration  are:  Mickey  Malone 

Billy  Prentice 
Eddie  Sullivan 


of  the  victim 
Shopping  Center 


•  Summary:  Robert  Gill,  a  prominent  local  businessman  was  found  dead  behind  his  Crestview  home 
this  morning.  Detective  Lt.  Mark  Moody  of  the  Hilltown  precinct  reported  that  Mr.  Gill  had 
apparently  been  assaulted  when  leaving  his  home  to  play  golf  early  this  morning.  He  was  struck 

on  the  head  over  the  left  eye  and  fell  down  a  flight  of  stairs  leading  from  a  second  story  deck  at  the 
rear  of  the  house.  The  preliminary  coroner’s  report  concluded  that  death  was  caused  by  injuries 
sustained  from  the  fall  and  not  from  the  blow  to  the  head.  The  report  estimated  that  Mr.  Gill’s 
death  occurred  between  6:30  and  7:00  AM.  Lt.  Moody  would  neither  confirm  nor  deny  rumors  that 
Mr.  Gill  had  been  robbed.  “We’re  following  all  leads.  That’s  all  I  have  to  say  for  now,”  said  Lt.  Moody. 

•  Team  Objective:  Collaborate  on  the  detailed  murder  information  and  develop  a  team  concensus 

on  who  killed  Mr.  Gill 


Independent  Variables 


NAV^AIR 


>  Collaboration  Mode  (face-to-face  vs  asynchronous,  distributed) 

*  Face-to-Face  =  team  interacts  synchronously  with  each  other  through  speech 

*  Asynchronous,  Distributed  =  team  interacts  with  each  other  at  different  times 

and  from  different  locations  through  a  text  based  web  forum 


>  Knowledge  Distribution  (homogeneous  vs  heterogeneous) 

*  Homogeneous  =  the  members  of  the  team  have  all  the  murder 

mystery  knowledge  in  common 

*  Heterogeneous  =  the  members  of  the  team  have  some  murder 

mystery  knowledge  in  common  and  some  uniquely 
held  murder  mystery  knowledge 
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N  A 


V^AIR 

COGNITIVE  AND  AUTOMATION 
RESEARCH  LAB  (CARL) 


Face-to-Face 
Collaboration  Area 


Asynchronous ,  Distributed 
Collaboration  Stations 


Experimenter ’s 
Station 


Electronic  Card  Wall  Collaboration  Tool  (Ew 


**  Produced  by  MIT  under  ONR  CKM  program 


Experience.  &.  Capabilities 

•  Over  25  Years  Experience  in 
Decision  Making  /  Automation  Research 


•  Member  National  &  International 
Research  Panels 


•  Recent  efforts;.  CASC,  Agent 
Learning,  ADSS,  ANGEL,  SCC 


•Tools:  local  web  server,  Pathfinder, 
Agent  development  toolsets,  Statistica 


•  Joint  efforts  (e.g.  NAVAIR  TSD,  JFCO 


Potential  Application s 


Users 


Exchange  Module 


s» 


Newsview  Module  Workspace  Module 

-  :  i  &  '  "  SJii.  01ar 

-  Mlfe  <4>  4  -I 

-i  jpjrp 

-  EiU 

-3  j  n 

is  *  m  s 

£5*  M.  EF-HFbK'B"  pa 


Improved  pilot  situational  awareness 
resulting  in  timely  mission  performan 
and  a  significant  decrease  in  aircraft  / 
aircrew  accidents 


•  More  timely  and  accurate  mission 
decisions  based  on  current  informatio 
achieved  through  asynchronous, 
distributed  collaboration  tools 
(C2  down  to  individual  warfighter) 


Improved  mission  planning  and 
execu^^i  through  networked 
asynchronous,  distributed  team 
collaboration  tools. 


EXPERIMENTAL  DESIGN 

( 2x2  randomized  factorial) 


NAV^AIR 


Knowledge  Distribution 


Collaboration  Face-to-Face 
Mode  <sPeech> 

Asynchronous, 

Distributed 

(text) 


Homogeneous 

Heterogeneous 

Gp  1 

Gp  15 

* 

* 

* 

* 

Gp  7 

Gp  21 

Gp  8 

Gp  22 

* 

* 

* 

* 

Gp  14 

Gp  28 

Phase  I 

•  28  groups  total 

•  3  subjects  /  group 

•  84  subjects  total 


DEPENDENT  VARIABLES: 


-  Forum  text  and  face-to-face  audio  /  video  recordings  including  time  stamp  per  response 
(I.e.  text  and  speech) 

-  Thinking  Aloud  Protocol  (concurrent  verbalizations)  for  asynchronous,  distributed  teams 

-  Total  time  to  successfully  complete  the  problem-solving  task  (time  from  the  beginning  of 
the  task  until  task  completion) 

-  Collaboration  Maps  (post  session  -  subjects  construct  a  map  of  their  view  of  the  stages  & 
cognitive  process  states  of  team  collaboration) 


-  Subjective  Questionnaire  -  measuring  expertise,  trust  between  team  members,  and  general 
collaboration  opinions  among  members  10 


TYPES  OF  DA  TA  ANAL  YSES 
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•  Verbal  Protocol  Communication  Analyses-  identification  of  collaboration  stages  and  cognitive 
process  states  compared  across  collaboration  mode  and  knowledge  distribution.  Compare  results 
to  preliminary  structural  model  of  collaboration. 

•  Transition  State  Diagrams  -  representation  of  the  dynamic  team  collaborative  behavior  between 
collaboration  stages  and  between  cognitive  process  states  within  each  collaboration  stage  compared 
across  collaboration  mode  and  knowledge  distribution.  Compare  results  to  preliminary 
structural  model  of  collaboration. 


•  Parametric  statistics  for  analyzing  time,  and  frequency  within  each  collaboration  stage  and 

cognitive  process  state  across  collaboration  mode  and  knowledge  distribution  conditions.  Also  used 
for  analyzing  total  time  to  complete  task  and  questionnaire  data. 


Collaboration  Maps  -  determine  the  degree  of  convergence  between  individual  mental  model ‘s 
regarding  collaboration  stages  and  cognitive  processes.  In  addition,  compare  how  an  indiv^ual 
thinks  a  group  makes  a  decision  in  a  collaborative  setting  and  how  the  group  actually  performs. 


Results 
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Mean  %  Time  by  Collaboration  Stages 

Collaboration  Mode  (CM)  vs.  Knowledge  Distribution  (KD) 


Team  Knowledge  Base  Construction: 

CM:  F  =5.51323,  p  =  0.027449 
KD:  F  =  L70682,  p  =  0.203780 
CMKD:  F  =6.02627,  p  =  0.02 1724 
N  =  84 
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Collaborative  Team  Problem  Solving: 

CM:  F  =6.944,  p  =  0.014504 
KD:  F  =2.010,  p=  0.169097 
CM'KD:  F  =  1.876,  p  =  0.183471 
N  =  84 

Collaboration  Mode/Knowledge  Distribution: 


Team  Consensus: 

CM:  F  =0.51854,  p  =  0.478422 
KD:  F  =3.43855,  p  =  0.07602 
CM*KD:  F  =0.82967,  p  =  0.371425 
N  =  84 


□  Face  to  Face  Homogeneous 

□  Asynchronous,  Distributed  Homogeneous 


|Face  to  Face  Heterogeneous 
|  Asynchronous,  Distributed  Heterogeneous 


100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20% 

10% 

0% 


Team  Knowledge 
Base  Construciton 


Collaborative  Team 
Problem  Solving 


Team 

Consensus 


Outcome  Evaluation 
&Revision 


=  significantly  different 


Collaboration  Stages 


*Ad  justed:  Each  team's  time  per  stage  was 
calculated  by  dividing  the  utterance  time  for  each 
stage  by  the  total  utterance  time  for  the  team. 


Results 
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Mean  %  Time  By  Cognitive  Process  States: 
Team  Knowledge  Construction  (TK) 

Wilks  lambda  =  0.402182,  F  =  3.158681,  p  =  0.022010 


®  Face  to  Face 
□  As  ynchr  o  no  us ,  Dis  trib  ut  e  d 


IP:  Information  Processing 
KB:  Knowledge  Building 


Understanding  Goal  Establishing  Data  Filtering  Transfer  Problem 

the  Problem  Definition  Team  Trust  (Pdf)  Conventions  Definition 
(Pup)  (Pgd)  (Pet)  (Ptck)  (KBpd) 

**  =  significantly  different  Process  States 


Individual  Team 

Knowledge  Knowledge 
(KBik)  (IKBtk) 


Results 
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Mean  %Time  By  Cognitive  Process  States: 
Collaborative  Team  Problem  Solving  (TPS ) 

Wilks  larrtxia= 0.00033  F= 9605.498  p= 0.000000 


q  Face  to  Face 
[]  Asynchronous,  E)istributed 


MC:  Meta-cognitive 
P:  information  Processing 
KB:  Knowledge  Building 


(MCtp) 

**  =  significant 


Analysis 

(Pica) 


Conventions 

(Ptcp) 


Alternatives 

(Psa) 


(Pde) 


Understanding 

(KBsu) 


Knowledge 

(KBck) 


NAV^AIR 
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Results 


NAV^AIR 


Summary  Conclusions 
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^  Both  face-to-face  and  asynchronous,  distributed  teams  demonstrated  behavior  that  supports  the 
existence  of  Team  Knowledge  Base  Construction,  Collaborative  Problem  Solving  and 
Team  Consensus  stages  during  collaborative  problem  solving.  Phase  I  data  showed  that  the  stages 
are  task  dependent  as  the  Outcome  Evaluation  &  Revision  stage  was  not  used 

The  significant  cognitive  process  states  were  Understanding  the  Problem,  Team  Knowledge 
development,  Conventions  for  Transferring  Meaning,  and  developing  Solution  Alternatives 

❖  Knowledge  Distribution  (homogeneous  /  heterogeneous  information)  did  not  make  any 
difference  on  the  time  spent  in  each  collaboration  stage 

❖  Asynchronous,  Distributed  teams  spent  more  time  in  Collaborative  Problem  Solving  stage 
than  face-to-face  teams.  Appears  to  be  more  difficult  to  solve  collaborative  problems  with  these 
types  of  teams  even  though  collaboration  environment  is  inherently  more  structured.  Need  to 
determine  why  these  teams  spend  more  time  so  collaborative  problem  solving  can  be  facilitated 

3*  Face-to-Face  teams  used  conventions  to  transfer  meaning  (e.g.  yellow  stickers,  maps)  whereas 
asynchronous,  distributed  teams  did  not  use  conventions.  Need  ways  to  easily  create  conventions 
with  asynchronous,  distributed  teams  (area  for  agent  support) 

0  Face-to-Face  teams  demonstrated  mostly  a  linear  path  between  team  knowledge  construction, 
team  problem  solving  and  team  consensus  whereas  asynchronous,  distributed  teams  sho^iqd  a 
non-linear  path  with  feedback  loops  (area  for  agent  support) 


Potential  Applications 
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Users 


Benefit 


•  More  timely  and  accurate  mission 
decisions  based  on  current  information 
achieved  through  asynchronous, 
distributed  collaboration  tools 
(C2  down  to  individual  warfighter) 


•  Improved  mission  planning  and 
execution  through  networked 
asynchronous,  distributed  team 
collaboration  tools. 
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